Kernel principal component analysis from a data modelling point of view

I show an example of how the principal component analysis method can be used for solving a specific computer vision problem—the one of fitting an ellipse to a set of points. In the example, the feature map is given (postulated) as part of the problem specification. More generally, the feature map in the kernel principal component analysis has the interpretation of the model class in data modelling problems. This interpretation relates the kernel selection problem in machine learning to the model selection problem in system identification. Data fitting by a second order model Consider the second order model B(A,b,c) := {d ∈ R | dAd + bd + c = 0} ⊂ R, with A = A (1) and define a parameter vector θ := col(a11,a12,a22,b1,b2,c) ∈ R. Any θ ∈ R6 corresponds to a second order model in R2, defined by (1), and vice verse, to any second order model in R2 there is a nonunique parameter vector θ ∈ R6, such that the model is (1). It can be verified that B(A,b,c) = {d ∈ R | θΦ(d) = 0} =: B(θ), where Φ(d) := col(d1d1,2d1d2,d2d2,d1,d2,1). (2) Note that B(θ) = B(αθ) for any α 6= 0, so that the parameter θ is unique only up to a multiplication by a nonzero constant. The considered data fitting problem is to find a second order model that best matches a set of given points D = {d, . . . ,d} ⊂ R in the sense of minimization of the fitting criterion